Videoconferencing enables individuals located remote from each other to have face-to-face meetings on short notice using audio and video telecommunications. A videoconference may involve as few as two sites (point-to-point) or several sites (multi-point). A single participant may be located at a conferencing site or there may be several participants at a site, such as at a conference room. Videoconferencing may also be used to share documents, information, and the like.
Participants in a videoconference interact with participants at other sites via a videoconferencing endpoint. An endpoint is a terminal on a network, capable of providing real-time, two-way audio/visual/data communication with other terminals or with a multipoint control unit (MCU, discussed in more detail below). An endpoint may provide speech only, speech and video, or speech, data and video communications, etc. A videoconferencing endpoint typically comprises a display unit on which video images from one or more remote sites may be displayed. Example endpoints include POLYCOM® RPD® and HDX® series, each available from Polycom, Inc. (POLYCOM, RPD, and HDX are registered trademarks of Polycom, Inc.). Such an endpoint can be referred as audio-video endpoint (AVEP). The videoconferencing endpoint sends audio, video, and/or data from a local site to the remote site(s) and displays video and/or data received from the remote site(s) on a screen.
Video images displayed on a screen at a videoconferencing endpoint may be arranged in a layout. The layout may include one or more segments for displaying video images. A segment is a portion of the screen of a receiving endpoint that is allocated to a video image received from one of the sites participating in the session. For example, in a videoconference between two participants, a segment may cover the entire display area of the screen of the local endpoint. Another example is a video conference between a local site and multiple other remote sites where the videoconference is conducted in switching mode, such that video from only one other remote site is displayed at the local site at a single time and the displayed remote site may be switched, depending on the dynamics of the conference. In contrast, in a continuous presence (CP) conference, a conferee at a terminal may simultaneously observe several other participants' sites in the conference. Each site may be displayed in a different segment of the layout, where each segment may be the same size or a different size. The choice of the sites displayed and associated with the segments of the layout may vary among different conferees that participate in the same session. In a continuous presence (CP) layout, a received video image from a site may be scaled down or cropped in order to fit a segment size.
An MCU may be used to manage a videoconference. An MCU is a conference controlling entity that may be located in a node of a network, in a terminal, or elsewhere. The MCU may receive and process several media channels, from access ports, according to certain criteria and distributes them to the connected channels via other ports. Examples of MCUs include the RMX® series of MCUs, available from Polycom Inc. (RMX series of MCUs are registered trademark of Polycom, Inc.). Some MCUs are composed of two logical units: a media controller and a media processor. A more thorough definition of an endpoint and an MCU may be found in the International Telecommunication Union (“ITU”) standards, such as but not limited to the H.320, H.324, and H.323 standards. Additional information regarding the ITU standards may be found at the ITU website www.itu.int.
To present a video image within a segment of a screen layout of a receiving endpoint (site), the entire received video image may be manipulated, scaled down and displayed, or a portion of the video image may be cropped by the MCU and displayed. An MCU may crop lines or columns from one or more edges of a received conferee video image in order to fit it to the area of a segment in the layout of the videoconferencing image. Another cropping technique may crop the edges of the received image according to a region of interest in the image, as disclosed in U.S. Pat. No. 8,289,371, the entire contents of which are incorporated herein by reference.
In a CP videoconferencing session, the association between sites and segments may be dynamically changed according to the activity taking part in the conference. In some layouts, one of the segments may be allocated to a current speaker, and other segments may be allocated to other sites, sites that were selected as presented conferees. The current speaker is typically selected according to certain criteria, such as the loudest speaker during a certain percentage of a monitoring period. The other sites (in the other segments) may include the previous speaker, sites with audio energy above the others, certain conferees required by management decisions to be visible, etc.
Another type of video conferencing system is media relay conferencing (MRC). In MRC, a Media Relay MCU (MRM) receives one or more streams from each participating Media Relay Endpoint (MRE). The MRM relays to each participating endpoint a set of multiple media streams received from other endpoints in the conference. Each receiving endpoint uses the multiple streams to generate the video CP image, according to a layout, as well as mixed audio of the conference. The CP video image and the mixed audio are played to MRE's user. An MRE can be a terminal of a conferee in the session which has the ability to receive relayed media from an MRM and deliver compressed media according to instructions from an MRM. A reader who wishes to learn more about an example of an MRC, MRM or an MRE is invited to read related patent and patent application, which were published and has the U.S. Pat. No. 8,228,363 and U.S. patent application Ser. No. 13/487,703 that are incorporated herein by reference. Along the description, the terms endpoint may represent also an MRE or AVEP.
In some MRC systems, a transmitting MRE sends its video image in two or more streams; each stream can be associated with different quality level. The qualities may differ in frame rate, resolution and/or signal to noise ratio (SNR), etc. In a similar way each transmitting MRE may sends its audio in two or more streams that may differ from each other by the compressing bit rate, for example. Such a system can use the plurality of video streams to provide different segment sizes in the layouts, different resolution used by each receiving endpoint, different quality, etc. Further, the plurality of streams can be used for overcoming packet loss.
For MRC video, for example, the quality can be expressed in number of domains, such as temporal domain (frames per second, for example), spatial domain (HD versus CIF, for example), and/or in quality (sharpness, for example). Video compression standards, for example, that can be used for multi quality streams are H.264 AVC, H.264 annex G (SVC), MPEG-4, etc. More information on compression standards such as H.264, MPEG-4, can be found at the ITU Website www.itu.int, or at www.mpeg.org.
Today a verity of common communication and computing devices can be used as video conferencing endpoints. Those devices have video camera, screen, microphone and speakers and can be used as videoconferencing endpoint. For example, smartphones, tablets, notebooks, etc., with or without a touchscreen, can participant in a video conferencing session. Some of the new devices offer new human interface tools. Tools like: scroll, pulling images, etc.
In addition there are few trends in the video communication. One trend in the market is improving the quality of the video. High Definition (HD) videoconferencing systems become more and more popular. In HD system video can be presented in high resolution (1920×1080 pixels) and in high frame rate (60 frames per second), for example. Consequently, HD videoconferencing system requires more computing resources and network resources.
Another trend in the video communication market is video-on-demand (VOD) that is served by a video services provider, such as but not limited to a cable TV (CTV) operator or an Internet TV service provider or an Internet service provider (ISP) that delivers video on demand services such as YouTube for example. Along the discloser the term VOD services, CTV, ISP of video streaming, can be used interchangeably. In such embodiments a TV receiver such as set top box, a computing device with communication capabilities over in IP network, or a mobile phone can be used as a video terminal. VOD services allow a user of a video terminal to communicate with the CTV operator and request to obtain a certain TV program. In many cases the user may jump from one program to the other looking for an interesting program. Usually jumping is done between similar stations such as sport stations, or news, or action, etc. While jumping from one channel to other the user may miss information such as a touchdown that is currently broadcast by another channel, for example.
Adapting videoconferencing technique to the new trends of computing devices and video communication would be beneficial and be appreciate by users of those systems.